User interfaces and associated methods

ABSTRACT

An apparatus and method for enabling the provision of differential haptic feedback to a user based on a particular degree of spatial interaction between a plurality of different user interface elements of a graphical user interface.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims priority to U.S.application Ser. No. 13/720,228, filed Dec. 19, 2012, the entirecontents of which are hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to user interfaces and user interfaceelements, associated methods, computer programs and apparatus. Certaindisclosed aspects/embodiments relate to portable electronic devices, inparticular, so-called hand-portable electronic devices which may behand-held in use (although they may be placed in a cradle in use). Suchhand-portable electronic devices include so-called Personal DigitalAssistants (PDAs), mobile telephones, smartphones and other smartdevices, and tablet PCs.

The portable electronic devices/apparatus according to one or moredisclosed aspects/embodiments may provide one or more audio/text/videocommunication functions (e.g. tele-communication, video-communication,and/or text transmission (Short Message Service (SMS)/Multimedia MessageService (MMS)/emailing) functions), interactive/non-interactive viewingfunctions (e.g. web-browsing, navigation, TV/program viewing functions),music recording/playing functions (e.g. MP3 or other format and/or(FM/AM) radio broadcast recording/playing), downloading/sending of datafunctions, image capture function (e.g. using a (e.g. in-built) digitalcamera), and gaming functions.

BACKGROUND

It is common for electronic devices to provide a user interface (e.g. agraphical user interface). A user interface may enable a user tointeract with an electronic device, for example, to open applicationsusing application icons, enter commands, to select menu items from amenu, or to enter characters using a virtual keypad.

The listing or discussion of a prior-published document or anybackground in this specification should not necessarily be taken as anacknowledgement that the document or background is part of the state ofthe art or is common general knowledge. One or more aspects/embodimentsof the present disclosure may or may not address one or more of thebackground issues.

SUMMARY

In a first aspect there is provided an apparatus, the apparatuscomprising:

-   -   at least one processor; and    -   at least one memory including computer program code,    -   the at least one memory and the computer program code        configured, with the at least one processor, to cause the        apparatus to perform at least the following:    -   enable the provision of differential haptic feedback based on        the particular spatial interaction between a plurality of user        interface elements of a graphical user interface.

Such differential haptic feedback may allow the user torecognise/inspect a spatial interaction between a plurality of userinterface elements using the sense of touch. This may be particularlyuseful when the spatial interaction is obscured (e.g. either wholly orpartially by the user's finger/stylus or by one or more of the userinterface elements themselves).

The differential haptic feedback may (or may not) be additional hapticfeedback provided when the plurality of user interface elements arespatially interacting which is not provided when the plurality of userinterface elements are not interacting.

The apparatus may be configured to enable the provision of thedifferential haptic feedback in the spatial interaction region of theplurality of user interface elements. The apparatus may be configured toenable provision of a first haptic feedback in the spatial interactionregion of the plurality of user interface elements and a seconddifferent haptic feedback in the spatial region outside the interactionregion of the plurality of user interface elements. This may beconsidered to be location specific differential haptic feedback. Thismay allow a user to receive information relating to the spatialinteraction whether or not the user is interacting with the spatialinteraction region.

The apparatus may be configured to provide the differential hapticfeedback in locations which are not related to the spatial interactionregion. This may be considered to be global differential hapticfeedback.

A plurality of user interface elements may be considered to interactwhen they come into contact (e.g. abut or have contiguous edges and/orsurfaces) and/or when they overlap (e.g. lie or extend over and coverpart of; or have an area, extent or range in common).

The apparatus may be configured such that the haptic feedback isdependent on one or more of:

-   -   the particular degree of overlap between the plurality of user        interface elements (e.g. related to the area or volume of the        overlap); and    -   the particular degree of contact between the plurality of user        interface elements (e.g. related to the length of a line segment        or the area of a surface shared by contiguous boundaries of the        plurality of user interface element).

The apparatus may be configured to enable the provision of the hapticfeedback when the spatial interaction between the plurality of userinterface elements is changing. The apparatus may be configured toenable the provision of the haptic feedback when the spatial interactionbetween the plurality of user interface elements is not changing.

The apparatus may be configured to enable the provision of thedifferential haptic feedback in response to stylus being detected withinthe detection range of the graphical user interface. The apparatus maybe configured to enable the provision of the differential hapticfeedback in real time as the user is interacting with the user interfaceelements (e.g. when the user is providing input, such as stationary orsliding input to the graphical user interface). It will be appreciatedthat not providing the differential haptic feedback continuously mayreduce energy requirements.

An input may comprise a single-touch or multi-touch gesture. The inputcould, of course, not be touch input, but could be input using a mouse,a touchpad, or other separate and/or external input device, for example.

The apparatus may be configured to provide differential haptic feedbackprovided to one of the plurality of user interface elements based onhaptic feedback associated with the user interface elements when theuser interface elements are not spatially interacting. For example, ifthe plurality of user interface elements have a cloth texture, thedifferential haptic feedback may also have a cloth texture (e.g. arougher cloth texture). This may provide a more consistent and/orintuitive user experience.

The apparatus may be configured such that the differential hapticfeedback is dependent on which of the plurality of user interfaceelements overlies with which other spatially interacting user interfaceelement. This may provide tactile information to the user relating towhich user interface element is overlying which other user interfaceelements.

The apparatus may be configured to provide differential haptic feedbackby changing one or more of:

-   -   the waveform of the haptic feedback (i.e. the shape and/or        frequency of the haptic wave);    -   the consistency of haptic feedback (e.g. whether the haptic        feedback is provided intermittently or constantly);    -   the strength of haptic feedback (e.g. the amplitude of an        electro tactile wave).

It will be appreciated that these examples may be considered to changethe type of the haptic feedback.

The apparatus may be configured to determine the spatial interaction.

The spatial interaction may comprise a spatial interaction in at leastone of one, two and three dimensions.

At least one of the plurality of user interface elements may beselectable to actuate an associated function performable using anelectronic device. The function might include opening an application,opening a file, or entering or formatting textual characters. A textualcharacter may comprise a combination of one or more of a word, a lettercharacter (e.g. from the Roman, Greek, Arabic or Cyrillic alphabets), agraphic character (e.g. a sinograph, Japanese kana or Koreandelineation), a numeral, a phrase, a syllable, a diacritical mark, anemoticon, and a punctuation mark.

A said user interface element may be an object provided by the graphicaluser interface which can be manipulated by the user. Such an object maynot be used to perform a function of an electronic device. The spatialinteraction may result from manipulation (e.g. by the user) of one ormore of the plurality of user interface elements

The haptic feedback may be provided using one or more of: a touchscreen, an array of haptic actuators; piezo-transducers, electro-activepolymers, and electro tactile materials.

The apparatus may comprise one or more of a touch screen, an array ofhaptic actuators; piezo-transducers, electro-active polymers, andelectro tactile materials to provide the differential haptic feedback.The apparatus may comprise the graphical user interface to provide theplurality of user interface elements as display outputs.

The apparatus may be a portable electronic device, a laptop computer, amobile phone, a Smartphone, a tablet computer, a personal digitalassistant, a digital camera, a watch, a server, a non-portableelectronic device, a desktop computer, a monitor, a server, a wand, apointing stick, a touchpad, a touch-screen, a mouse, a joystick or amodule/circuitry for one or more of the same.

According to a further aspect, there is provided a method, the methodcomprising:

-   -   enabling the provision of differential haptic feedback based on        the particular spatial interaction between a plurality of user        interface elements of a graphical user interface.

According to a further aspect, there is provided a computer programcomprising computer program code, the computer program code beingconfigured to perform at least the following:

-   -   enable the provision of differential haptic feedback based on        the particular spatial interaction between a plurality of user        interface elements of a graphical user interface.

A computer program may be stored on a storage media (e.g. on a CD, aDVD, a memory stick or other non-transitory medium). A computer programmay be configured to run on a device or apparatus as an application. Anapplication may be run by a device or apparatus via an operating system.A computer program may form part of a computer program product.

According to a further aspect there is provided an apparatus comprising:

-   -   means for enabling configured to enable the provision of        differential haptic feedback based on the particular spatial        interaction between a plurality of user interface elements of a        graphical user interface

According to a further aspect there is provided an apparatus comprising:

-   -   an enabler configured to enable the provision of differential        haptic feedback based on the particular spatial interaction        between a plurality of user interface elements of a graphical        user interface.

According to a further aspect, there is provided an apparatus, theapparatus comprising:

-   -   at least one processor; and    -   at least one memory including computer program code,    -   the at least one memory and the computer program code        configured, with the at least one processor, to cause the        apparatus to perform at least the following:    -   enable the provision of differential haptic feedback associated        with a particular user interface element based on the layer of        the graphical user interface which the user interface element        occupies.

According to a further aspect, there is provided a method, the methodcomprising:

-   -   enabling the provision of differential haptic feedback        associated with a particular user interface element based on the        layer of the graphical user interface which the user interface        element occupies.

According to a further aspect, there is provided a computer programcomprising computer program code, the computer program code beingconfigured to perform at least the following:

-   -   enable the provision of differential haptic feedback associated        with a particular user interface element based on the layer of        the graphical user interface which the user interface element        occupies.

The present disclosure includes one or more corresponding aspects,embodiments or features in isolation or in various combinations whetheror not specifically stated (including claimed) in that combination or inisolation. Corresponding means and corresponding function units (e.g. adeterminer, an enabler) for performing one or more of the discussedfunctions are also within the present disclosure.

Corresponding computer programs for implementing one or more of themethods disclosed are also within the present disclosure and encompassedby one or more of the described embodiments.

The above summary is intended to be merely exemplary and non-limiting.

BRIEF DESCRIPTION OF THE FIGURES

A description is now given, by way of example only, with reference tothe accompanying drawings, in which:

FIG. 1 depicts an example apparatus embodiment comprising a number ofelectronic components, including memory and a processor;

FIG. 2 depicts an example apparatus embodiment comprising a number ofelectronic components, including memory, a processor and a communicationunit;

FIG. 3 depicts an example apparatus embodiment comprising a number ofelectronic components, including memory, a processor and a communicationunit;

FIG. 4 a-4 d depicts an example embodiment wherein the user interfaceelements are windows;

FIG. 5 a-5 d depicts a further example embodiment wherein the userinterface elements are icons;

FIG. 6 a-6 c depicts a further example embodiment wherein the userinterface elements can be manipulated by the user;

FIGS. 7 a-7 b illustrate an example apparatus in communication with aremote server/cloud;

FIG. 8 illustrates a flowchart according to an example method of thepresent disclosure; and

FIG. 9 illustrates schematically a computer readable medium providing aprogram.

DESCRIPTION OF EXAMPLE ASPECTS/EMBODIMENTS

It is common for an electronic device to have a user interface (whichmay or may not be graphically based) to allow a user to interact withthe device to enter and/or interact with information. For example, theuser may use a keyboard user interface to enter text, or use icons toopen applications. Some user interfaces include displays, such as touchscreens, which can display information to the user.

On a touch screen, when arranging multiple user interface elements (e.g.icons, windows) the user may only receive visual feedback of the objectsrelative positions, and this may be somewhat obscured by the fingerswhich are controlling the user interface elements.

This disclosure relates to providing differential haptic feedback basedon the spatial interaction between a plurality of user interfaceelements. This may allow the user to receive tactile information on thespatial interaction as well as or instead of visual information. Thedifferential haptic feedback may provide information on when items aretouching and/or overlapping each other and, in some cases, on the degreeto which the items are overlapping each other. It will be appreciatedthat by providing differential feedback, the user interface may moreaccurately replicate the touch sense a user would experience wheninteracting with physical objects. Therefore, embodiments may provide amore intuitive and immersive user experience.

Other embodiments depicted in the figures have been provided withreference numerals that correspond to similar features of earlierdescribed embodiments. For example, feature number 1 can also correspondto numbers 101, 201, 301 etc. These numbered features may appear in thefigures but may not have been directly referred to within thedescription of these particular embodiments. These have still beenprovided in the figures to aid understanding of the further embodiments,particularly in relation to the features of similar earlier describedembodiments.

FIG. 1 shows an apparatus 101 comprising memory 107, a processor 108,input I and output O. In this embodiment only one processor and onememory are shown but it will be appreciated that other embodiments mayutilise more than one processor and/or more than one memory (e.g. sameor different processor/memory types).

In this embodiment the apparatus 101 is an Application SpecificIntegrated Circuit (ASIC) for a portable electronic device with a touchsensitive display. In other embodiments the apparatus 101 can be amodule for such a device, or may be the device itself, wherein theprocessor 108 is a general purpose CPU of the device and the memory 107is general purpose memory comprised by the device.

The input I allows for receipt of signalling to the apparatus 101 fromfurther components, such as components of a portable electronic device(like a touch-sensitive display) or the like. The output O allows foronward provision of signalling from within the apparatus 101 to furthercomponents. In this embodiment the input I and output O are part of aconnection bus that allows for connection of the apparatus 101 tofurther components.

The processor 108 is a general purpose processor dedicated toexecuting/processing information received via the input I in accordancewith instructions stored in the form of computer program code on thememory 107. The output signalling generated by such operations from theprocessor 108 is provided onwards to further components via the outputO.

The memory 107 (not necessarily a single memory unit) is a computerreadable medium (solid state memory in this example, but may be othertypes of memory such as a hard drive, ROM, RAM, Flash or the like) thatstores computer program code. This computer program code storesinstructions that are executable by the processor 108, when the programcode is run on the processor 108. The internal connections between thememory 107 and the processor 108 can be understood to, in one or moreexample embodiments, provide an active coupling between the processor108 and the memory 107 to allow the processor 108 to access the computerprogram code stored on the memory 107.

In this example the input I, output O, processor 108 and memory 107 areall electrically connected to one another internally to allow forelectrical communication between the respective components I, O, 108,107. In this example the components are all located proximate to oneanother so as to be formed together as an ASIC, in other words, so as tobe integrated together as a single chip/circuit that can be installedinto an electronic device. In other examples one or more or all of thecomponents may be located separately from one another.

FIG. 2 depicts an apparatus 201 of a further example embodiment, such asa mobile phone. In other example embodiments, the apparatus 201 maycomprise a module for a mobile phone (or PDA or audio/video player), andmay just comprise a suitably configured memory 207 and processor 208.The apparatus in certain embodiments could be a portable electronicdevice, a laptop computer, a mobile phone, a Smartphone, a tabletcomputer, a personal digital assistant, a digital camera, a watch, aserver, a non-portable electronic device, a desktop computer, a monitor,a server, a wand, a pointing stick, a touchpad, a touch-screen, a mouse,a joystick or a module/circuitry for one or more of the same

The example embodiment of FIG. 2, in this case, comprises a displaydevice 204 such as, for example, a Liquid Crystal Display (LCD) ortouch-screen user interface. The apparatus 201 of FIG. 2 is configuredsuch that it may receive, include, and/or otherwise access data. Forexample, this example embodiment 201 comprises a communications unit203, such as a receiver, transmitter, and/or transceiver, incommunication with an antenna 202 for connecting to a wireless networkand/or a port (not shown) for accepting a physical connection to anetwork, such that data may be received via one or more types ofnetworks. This example embodiment comprises a memory 207 that storesdata, possibly after being received via antenna 202 or port or afterbeing generated at the user interface 205. The processor 208 may receivedata from the user interface 205, from the memory 207, or from thecommunication unit 203. It will be appreciated that, in certain exampleembodiments, the display device 204 may incorporate the user interface205. Regardless of the origin of the data, these data may be outputtedto a user of apparatus 201 via the display device 204, and/or any otheroutput devices provided with apparatus. The processor 208 may also storethe data for later use in the memory 207. The memory 207 may storecomputer program code and/or applications which may be used toinstruct/enable the processor 208 to perform functions (e.g. read,write, delete, edit or process data).

FIG. 3 depicts a further example embodiment of an electronic device 301,such as a tablet personal computer, a portable electronic device, aportable telecommunications device, a server or a module for such adevice, the device comprising the apparatus 101 of FIG. 1. The apparatus101 can be provided as a module for device 301, or even as aprocessor/memory for the device 301 or a processor/memory for a modulefor such a device 301. The device 301 comprises a processor 308 and astorage medium 307, which are connected (e.g. electrically and/orwirelessly) by a data bus 380. This data bus 380 can provide an activecoupling between the processor 308 and the storage medium 307 to allowthe processor 308 to access the computer program code. It will beappreciated that the components (e.g. memory, processor) of thedevice/apparatus may be linked via cloud computing architecture. Forexample, the storage device may be a remote server accessed via theinternet by the processor.

The apparatus 101 in FIG. 3 is connected (e.g. electrically and/orwirelessly) to an input/output interface 370 that receives the outputfrom the apparatus 101 and transmits this to the device 301 via data bus380. Interface 370 can be connected via the data bus 380 to a display304 (touch-sensitive or otherwise) that provides information from theapparatus 101 to a user. Display 304 can be part of the device 301 orcan be separate. The device 301 also comprises a processor 308configured for general control of the apparatus 101 as well as thedevice 301 by providing signalling to, and receiving signalling from,other device components to manage their operation.

The storage medium 307 is configured to store computer code configuredto perform, control or enable the operation of the apparatus 101. Thestorage medium 307 may be configured to store settings for the otherdevice components. The processor 308 may access the storage medium 307to retrieve the component settings in order to manage the operation ofthe other device components. The storage medium 307 may be a temporarystorage medium such as a volatile random access memory. The storagemedium 307 may also be a permanent storage medium such as a hard diskdrive, a flash memory, a remote server (such as cloud storage) or anon-volatile random access memory. The storage medium 307 could becomposed of different combinations of the same or different memorytypes.

FIGS. 4 a-4 d depicts an example embodiment of the apparatus depicted inFIG. 2 comprising a portable electronic communications device 401, e.g.such as a mobile phone, with a user interface comprising a touch screenuser interface 405, 404, a memory (not shown), a processor (not shown)and an antenna (not shown) for transmitting and/or receiving data (e.g.emails, textual messages, phone calls, information corresponding to webpages). The touch screen 405, 404 in this case is configured to providehaptic feedback using electrostatic tactile technology.

In this case, the apparatus/device is configured determine the spatialinteraction; and to enable the provision of differential haptic feedback(al least the signalling required to provide for the differential hapticfeedback) based on the particular spatial interaction between aplurality of user interface elements of a graphical user interface. Itwill be appreciated that certain embodiments of the apparatus need notnecessarily comprise the graphical user interface or the hapticactuators, for example, as long as the apparatus (e.g. processor/memory)are configured to provide the appropriate signalling to enable theprovision of the differential haptic feedback.

In this case, the apparatus is configured to provide global differentialhaptic feedback when the user is manipulating the user interfaceelements to change the spatial interaction in a region not related tothe interaction region (e.g. to alert the user that the spatialinteraction is changing regardless of where on the screen the user isinteracting), and also location specific differential haptic feedback toindicate the location and/or the extent of the spatial interaction whenthe spatial interaction between the plurality of user interface elementsis not changing (e.g. when the user is examining the spatial interactionbetween the plurality of user interface elements). In other embodiments,one of location specific and global differential haptic feedback may beprovided. It will be appreciated that the location specific differentialhaptic feedback may also be provided when the spatial interactionbetween the plurality of user interface elements is changing.

In the situation shown in FIG. 4 a, the user is moving first and secondwindow graphical user interface elements 411, 412 across the screenusing his fingers 491, 492. These window graphical user interfaceelements 411, 412 represent application instances such as text documentsand/or web browser windows. In this case, the apparatus/device isconfigured to provide global differential haptic feedback when thespatial interaction between the plurality of window graphical userinterface elements is changing. In FIG. 4 a, the graphical userinterface elements 411, 412 are spaced apart and so are not spatiallyinteracting on the touch screen 404, 405. In this case, when thegraphical user interfaces are not spatially interacting, no globaldifferential haptic feedback is provided.

As the user brings the graphical user interface elements together (bydragging using his fingers 491, 492 or other styli), the windowgraphical user interface elements 411, 412 touch (and thereby spatiallyinteract), as shown in FIG. 4 b. When the plurality of user interfaceelements 411, 412 are spatially interacting, the apparatus/device isconfigured to enable the provision of global differential hapticfeedback 421, 422 to the users fingers based on the particular spatialinteraction. In certain embodiments, this differential haptic feedback(with respect to a non-interacting configuration) may remain constantregardless of the degree of interaction. Also, in other embodiments, thedifferential haptic feedback may stop after the initial detectedinteraction.

However, in this case, the strength of the global differential hapticfeedback is dependent on the particular degree of overlap between theplurality of user interface elements 411, 412 (i.e. on the area ofoverlap area 431). In the case of contact, as shown in FIG. 4 b, theextent of overlap is small so the strength of the global differentialhaptic feedback 421, 422 is correspondingly small. In this case, whenthe user is manipulating the user interface elements by moving them, theapparatus/device is configured to provide global differential hapticfeedback 421, 422 at the position of the stylus/styli (which in thiscase is the user's finger/fingers 491, 492) as shown in FIG. 4 b. Thatis, the differential haptic feedback is provided in real time as theuser is interacting with the user interface elements.

It will be appreciated that the global differential haptic feedbackprovided is in addition to, and in certain embodiments distinct from,the haptic feedback provided when the user interface elements are notspatially interacting.

In this embodiments, when the user brings the window user interfaceelements together such that the overlap increases (as shown in FIG. 4c), the strength of the global differential haptic feedback 421, 422increases correspondingly (e.g. by increasing the amplitude of theelectro tactile waveform).

In this embodiment, when the user releases the window user interfaceelements 411, 412, they are configured to remain statically in position.When the user interface elements are static, the spatial interactionbetween the plurality of user interface elements is not changing so theglobal differential haptic feedback ceases. However, when the spatialinteraction between the plurality of user interface elements is notchanging the apparatus is configured to provide location specificdifferential haptic feedback at different locations on the touch screenuser interface. That is, the apparatus/device is configured to providefirst haptic feedback in the spatial interaction region 431 of theplurality of user interface elements 411, 412 and a second differenthaptic feedback in the spatial region outside the interaction region ofthe plurality of user interface elements 411, 412. In this case, thefirst and second haptic feedback is generated by providing differentwaveforms (e.g. corresponding to different textures) for the interactionregion 431 than for the portions of the user interface elements outsidethe interaction region.

In this case, by providing location specific differential spatial hapticfeedback which indicates which portion or portions of the plurality ofuser interface elements are interacting, the user can detect a differentsurface texture between the overlapping portions 431 of the userinterface elements and the non-overlapped portions of the user interfaceelements 411, 412 by moving his finger across the surface of the touchscreen. That is, in this case, the texture of the overlapping area 431of the two user interface elements is different to the haptic texture ofthe non-overlapping areas.

It will be appreciated that the device/apparatus may be configured toenable the provision of the differential haptic feedback only when astylus is detected within the detection range of the graphical userinterface.

FIG. 5 a shows a further example embodiment of the apparatus depicted inFIG. 2 comprising a portable electronic communications device 501, e.g.such as a mobile phone, with a user interface comprising a touch-screenuser interface 505, 504, a memory (not shown), a processor (not shown)and an antenna (not shown) for transmitting and/or receiving data (e.g.emails, textual messages, phone calls, information corresponding to webpages).

In this example, the display of the mobile phone is configured todisplay a number of icon user interface elements 511-514. Each icon userinterface element 511-514 is selectable to actuate an associatedfunction performable using the electronic device. In this case, thereare icon user interface elements corresponding to the functions ofnavigating to a home screen of the electronic device 511; opening agames application 512; opening a telephone call application 513; andopening a settings menu 514. In this case, each of the user interfaceelements can be manipulated by the user to provide for moving and/orresizing of the icons.

In this example, the user uses the telephone icon user interface elementmost frequently, and so wants it to be at the top and enlarged for easeof use. To do this, the user selects and moves the telephone icon userinterface element 513 and moves it towards the top using a dragginggesture using the touch screen graphical user interface. In thisembodiment, each user icon occupies a different layer within thegraphical user interface which dictates which user interface elementoverlies which other user interface elements when they overlap. In thiscase, the telephone user interface element 513 occupies a layer which isabove the layer occupied by the home screen user interface element 511.

In this case, the user interface is configured to provide each of theuser interface elements 511-514 with a different texture (e.g. even whenthey are not spatially interacting), so that a user can distinguishbetween the user interface elements by touch. In this case, thetelephone user interface element 513 is configured to have a sandy feel,whereas the home screen user interface element 511 is configured to havethe feeling of cloth. In this case, the haptic feedback is providedusing Electro Active Polymer technology. It will be appreciated thatother embodiments may use haptic actuators, piezo-transducers,electro-active polymers or other haptic technologies.

As the user brings the telephone user interface element 513 towards thehome screen user interface element 511 (as shown in FIG. 2 a) thetelephone user interface element 513 overlaps the home screen userinterface element 511. Because the telephone user interface element 513occupies a layer above the home screen user interface element 511, whenthe two user interface elements are overlapping, the telephone userinterface element 513 obscures the home screen user interface elementbecause it overlies the home screen user interface element 511.

In this case, when the telephone user interface element is spatiallyinteracting with the home screen user interface element by overlappingthe home screen user interface element, the apparatus/device isconfigured to enable the provision of location specific differentialhaptic feedback based on the particular spatial interaction between aplurality of user interface elements of the graphical user interface. Inthis case, the apparatus/device is configured to increase the roughnessof the texture of the overlying user interface element (which in thiscase is the telephone user interface element 513) in the spatiallyinteracting region (i.e. thereby providing a different first hapticfeedback in the spatial interaction region 531 of the plurality of userinterface elements than the second haptic feedback provided outside thespatial interaction region).

So in this example, the roughness of the sandy texture of the telephoneuser interface is increased in the portion which is overlapping the homescreen user interface element. It will be appreciated, that if the homescreen user interface element was on top of/overlying the telephone userinterface element, the overlapping spatial interaction region 531 wouldhave a cloth feel, the texture of the cloth being rougher than the feelof the portion of the home screen user interface element which isoutside the interaction region of the spatial region of the plurality ofuser interface elements.

As the user moves the telephone user interface element such that thedegree (area) of overlap increases, the user detected roughness of theoverlapping spatial interaction region increases.

In the situation depicted in FIG. 5 d, the user has finished positioningthe telephone user interface element and has enlarged the telephone userinterface element such that it completely overlies the home screen userinterface element 511. Although the underlying home screen userinterface element 511 is completely obscured (the dotted lines in FIGS.5 c and 5 d are for guidance only), by providing differential hapticfeedback in the respective regions of overlap/non-overlap (both of whichobscure underlying objects), the user can determine where the homescreen user interface element is located using touch (e.g. even if, asin this case, the home screen user interface element is visuallyobscured by an overlying user interface element). That is, a differentfeedback would be provided within the perimeter defined by 531 andoutside the perimeter 531 but within the area of the user interfaceelement 513.

It will be appreciated that embodiments which are configured such thatuser interface elements occupy different layers may enable the provisionof differential haptic feedback associated with a particular userinterface element based on the layer of the graphical user interfacewhich the user interface element occupies. For example, the userinterface elements towards the top may have a smoother texture than theuser interface elements towards the bottom. This may allow the user todetermine which layer the particular user interface element occupieswithout having to move the user interface elements around such that theyoverlap (to get a visual indication).

FIG. 6 shows a further example embodiment of the apparatus depicted inFIG. 2 comprising a electronic device 601, e.g. such as a desktopcomputer, comprising: a user interface having a display 604 and a mouse605 (unlike the touch screen user interfaces of the embodiments offigured 4 and 5); a memory (not shown); a processor (not shown) and annetwork connection (not shown) for transmitting and/or receiving data(e.g. emails, textual messages, phone calls, information correspondingto web pages). This may allow the user to find an obscured userinterface element without having to move other user interface elementsaround.

In this case, the user is using a home planning application of theelectronic device to arrange furniture in a virtual room. In this casethe virtual room includes icon user interface elements which represent atwo-seat sofa 611, a three-seat sofa 613, a table 612, a desk 614 and achair 615.

In this case, the user wants to have a larger table so, using the mouse,moves the cursor 695 to select the table user interface element 612. Theuser then enlarges the table by dragging the left edge of the table userinterface element to the left (as shown in FIG. 6 b).

As the user table user interface element 612 is enlarged it spatiallyinteracts with the two seat sofa user interface element 611. In thiscase, the device/apparatus is configured to enable the provision ofglobal differential haptic feedback 631 based on the particular spatialinteraction between a plurality of user interface elements 611-615 of agraphical user interface.

In this case, the device/apparatus is configured to provide globaldifferential haptic feedback 621 as the spatial interaction between theplurality of user interface elements are changing. In this case, themouse 605 is enabled to provide the global differential haptic feedback621 by being configured to vibrate with a frequency which isproportional to the area of overlap. In the situation depicted in FIG. 6b, when the user is enlarging the table user interface element 612 suchthat it spatially interacts with the two seat sofa user interfaceelement 611, the mouse 605 is configured to provide vibration hapticfeedback 621 with a low frequency because the degree of overlap 631 isrelatively small.

When the user has enlarged the table user interface element 612 furthersuch that the degree of overlap is larger (as shown in FIG. 6 c), thefrequency of the mouse vibration differential haptic feedback 621 isincreased. In this embodiment, when the user has completed manipulatingthe user interface elements, the apparatus/device is configured to stopproviding the global differential haptic feedback.

Like the embodiment of FIG. 4, this embodiment is also configured toprovide location specific haptic feedback. That is, the apparatus/deviceis configured such that the mouse vibrates in a different way when thecursor is positioned within the spatial interaction region 631 of theplurality of user interface elements 611-615 than when the cursor isconfigured to be in a position outside the interaction region of theplurality of user interface elements. In this way, the user can explorethe location and extent of the spatial interaction using mouse movementaround the display.

If will be appreciated that in other embodiments, the apparatus/devicemay be configured to provided differential haptic feedback in differentways. For example, the apparatus/device may be configured to change theconsistency of the haptic feedback. For example, when the degree ofspatial interaction is small, the device/apparatus may be configured toprovide intermittent haptic feedback with small bursts of hapticfeedback separated by large gaps. As the degree of spatial interactionincreases, the apparatus/device may be configured to increase theduration of the haptic feedback bursts and/or decrease the duration ofthe separating gaps. When there is a large spatial interaction, thedevice/apparatus may be configured to provide the haptic feedbackcontinuously.

It will be appreciated that other embodiments may allow the user tomanipulate user interface elements in three dimensions. In this case,the degree of spatial interaction may be measured in terms of volume.

FIG. 7 a shows that an example embodiment of an apparatus incommunication with a remote server. FIG. 7 b shows that an exampleembodiment of an apparatus in communication with a “cloud” for cloudcomputing. In FIGS. 7 a and 7 b, apparatus 701 (which may be apparatus101, 201 or 301 is in communication with a display 704). Of course, theapparatus 701 and display 704 may form part of the sameapparatus/device, although they may be separate as shown in the figures.The apparatus 701 is also in communication with a remote computingelement. Such communication may be via a communications unit, forexample. FIG. 7 a shows the remote computing element to be a remoteserver 795, with which the apparatus may be in wired or wirelesscommunication (e.g. via the internet, Bluetooth, a USB connection, orany other suitable connection as known to one skilled in the art). InFIG. 7 b, the apparatus 701 is in communication with a remote cloud 796(which may, for example, by the Internet, or a system of remotecomputers configured for cloud computing). It may be that the functionsassociated with the user interface elements are stored at the remotecomputing element 795, 796) and accessed by the apparatus 701 fordisplay 704. The enabling may be performed at the remote computingelement 795, 796. The apparatus 701 may actually form part of the remotesever 795 or remote cloud 796. In such embodiments, enablement of theprovision of feedback may be conducted by the server or in conjunctionwith use of the server.

FIG. 8 illustrates the process flow according to an example embodimentof the present disclosure. The process comprises enabling 881 theprovision of differential haptic feedback based on the particularspatial interaction between a plurality of user interface elements of agraphical user interface.

FIG. 9 illustrates schematically a computer/processor readable medium(900) providing a program according to an embodiment. In this example,the computer/processor readable medium is a disc such as a DigitalVersatile Disc (DVD) or a compact disc (CD).

In other embodiments, the computer readable medium may be any mediumthat has been programmed in such a way as to carry out the functionalityherein described. The computer program code may be distributed betweenthe multiple memories of the same type, or multiple memories of adifferent type, such as ROM, RAM, flash, hard disk, solid state, etc.

Any mentioned apparatus/device/server and/or other features ofparticular mentioned apparatus/device/server may be provided byapparatus arranged such that they become configured to carry out thedesired operations only when enabled, e.g. switched on, or the like. Insuch cases, they may not necessarily have the appropriate softwareloaded into the active memory in the non-enabled (e.g. switched offstate) and only load the appropriate software in the enabled (e.g. onstate). The apparatus may comprise hardware circuitry and/or firmware.The apparatus may comprise software loaded onto memory. Suchsoftware/computer programs may be recorded on the samememory/processor/functional units and/or on one or morememories/processors/functional units.

In some embodiments, a particular mentioned apparatus/device/server maybe pre-programmed with the appropriate software to carry out desiredoperations, and wherein the appropriate software can be enabled for useby a user downloading a “key”, for example, to unlock/enable thesoftware and its associated functionality. Advantages associated withsuch embodiments can include a reduced requirement to download data whenfurther functionality is required for a device, and this can be usefulin examples where a device is perceived to have sufficient capacity tostore such pre-programmed software for functionality that may not beenabled by a user.

Any mentioned apparatus/circuitry/elements/processor may have otherfunctions in addition to the mentioned functions, and that thesefunctions may be performed by the sameapparatus/circuitry/elements/processor. One or more disclosed aspectsmay encompass the electronic distribution of associated computerprograms and computer programs (which may be source/transport encoded)recorded on an appropriate carrier (e.g. memory, signal).

Any “computer” described herein can comprise a collection of one or moreindividual processors/processing elements that may or may not be locatedon the same circuit board, or the same region/position of a circuitboard or even the same device. In some embodiments one or more of anymentioned processors may be distributed over a plurality of devices. Thesame or different processor/processing elements may perform one or morefunctions described herein.

The term “signalling” may refer to one or more signals transmitted as aseries of transmitted and/or received electrical/optical signals. Theseries of signals may comprise one, two, three, four or even moreindividual signal components or distinct signals to make up saidsignalling. Some or all of these individual signals may betransmitted/received by wireless or wired communication simultaneously,in sequence, and/or such that they temporally overlap one another.

With reference to any discussion of any mentioned computer and/orprocessor and memory (e.g. including ROM, CD-ROM etc), these maycomprise a computer processor, Application Specific Integrated Circuit(ASIC), field-programmable gate array (FPGA), and/or other hardwarecomponents that have been programmed in such a way to carry out theinventive function.

The applicant hereby discloses in isolation each individual featuredescribed herein and any combination of two or more such features, tothe extent that such features or combinations are capable of beingcarried out based on the present specification as a whole, in the lightof the common general knowledge of a person skilled in the art,irrespective of whether such features or combinations of features solveany problems disclosed herein, and without limitation to the scope ofthe claims. The applicant indicates that the disclosedaspects/embodiments may consist of any such individual feature orcombination of features. In view of the foregoing description it will beevident to a person skilled in the art that various modifications may bemade within the scope of the disclosure.

While there have been shown and described and pointed out fundamentalnovel features as applied to example embodiments thereof, it will beunderstood that various omissions and substitutions and changes in theform and details of the devices and methods described may be made bythose skilled in the art without departing from the spirit of thedisclosure. For example, it is expressly intended that all combinationsof those elements and/or method steps which perform substantially thesame function in substantially the same way to achieve the same resultsare within the scope of the disclosure. Moreover, it should berecognized that structures and/or elements and/or method steps shownand/or described in connection with any disclosed form or embodimentsmay be incorporated in any other disclosed or described or suggestedform or embodiment as a general matter of design choice. Furthermore, inthe claims means-plus-function clauses are intended to cover thestructures described herein as performing the recited function and notonly structural equivalents, but also equivalent structures. Thusalthough a nail and a screw may not be structural equivalents in that anail employs a cylindrical surface to secure wooden parts together,whereas a screw employs a helical surface, in the environment offastening wooden parts, a nail and a screw may be equivalent structures.

1. An apparatus, the apparatus comprising: at least one processor; andat least one memory including computer program code, the at least onememory and the computer program code configured, with the at least oneprocessor, to cause the apparatus to perform at least the following:enable the provision of differential haptic feedback based on thespatial interaction between a plurality of user interface elementsassociated with an electronic device.
 2. The apparatus of claim 1,wherein the apparatus is configured to enable the provision of a firsthaptic feedback in the spatial interaction region of the plurality ofuser interface elements and a second different haptic feedback in thespatial interaction region of the plurality of user interface elements.3. The apparatus of claim 1, wherein the apparatus is configured suchthat the haptic feedback is dependent on one or more of: the particulardegree of overlap between the plurality of user interface elements; andthe particular degree of contact between the plurality of user interfaceelements.
 4. The apparatus of claim 1, wherein the apparatus isconfigured to enable the provision of the differential haptic feedbackwhen one or more of a stylus and finger is detected within the detectionrange of a graphical user interface associated with the electronicdevice.
 5. The apparatus of claim 1, wherein the apparatus is configuredto provide differential haptic feedback provided to one of the pluralityof user interface elements based on haptic feedback associated with theuser interface elements when the user interface elements are notspatially interacting.
 6. The apparatus of claim 1, wherein theapparatus is configured such that the differential haptic feedback isdependent on which of the plurality of user interface elements overlieswith which other spatially interacting user interface element.
 7. Theapparatus of claim 1, wherein the apparatus is configured to providedifferential haptic feedback by changing one or more of: the waveform ofthe haptic feedback; duration of haptic feedback; the consistency ofhaptic feedback; the strength of haptic feedback.
 8. The apparatus ofclaim 1, wherein the apparatus is configured to determine the spatialinteraction.
 9. The apparatus of claim 1, wherein the spatialinteraction comprises a spatial interaction in at least one of one, twoand three dimensions.
 10. The apparatus of claim 1, wherein at least oneof the plurality of user interface elements is configured to beselectable to actuate an associated function performable using a or theelectronic device.
 11. The apparatus of claim 1 wherein at least one ofthe plurality of user interface elements is an object which can bemanipulated by the user of the electronic device but which cannot beused to perform a function with the electronic device.
 12. The apparatusof claim 1, wherein the spatial interaction is enabled usingmanipulation of one or more of the plurality of user interface elements13. The apparatus of claim 1, wherein the apparatus is configured toenable the provision of the differential haptic feedback in real time asthe user is interacting with the user interface elements.
 14. Theapparatus of claim 1, wherein the haptic feedback is provided using oneor more of: an array of haptic actuators; piezo-transducers,electro-active polymers, and electro tactile materials.
 15. Theapparatus of claim 1, wherein the apparatus comprises one or more of: anarray of haptic actuators; piezo-transducers, electro-active polymers,and electro tactile materials to provide the differential hapticfeedback.
 16. The apparatus of claim 1, wherein the apparatus comprisesa graphical user interface configured to provide the plurality of userinterface elements as display outputs.
 17. The apparatus of claim 1,wherein the apparatus is the electronic device, a user interface, agraphical user interface, a portable electronic device, a portabletelecommunications device, a laptop computer, a mobile phone, anaudio/video player, a Smartphone, a tablet computer, a personal digitalassistant, a digital camera, a watch, a server, a non-portableelectronic device, a desktop computer, a monitor, a server, a wand, apointing stick, a touchpad, a touch-screen, a mouse, a joystick or amodule/circuitry for one or more of the same.
 18. The apparatus of claim1, wherein one or more of spatial interaction input to the plurality ofuser interface elements and differential haptic feedback based on thespatial interaction between the plurality of user interface elements isrespectively provided by or to a finger, a stylus, a keyboard, anexternal input device to the electronic device, the electronic device, auser interface, a graphical user interface, a portable electronicdevice, a portable telecommunications device, a laptop computer, amobile phone, an audio/video player, a Smartphone, a tablet computer, apersonal digital assistant, a digital camera, a watch, a server, anon-portable electronic device, a desktop computer, a monitor, a server,a wand, a pointing stick, a touchpad, a touch-screen, a mouse, ajoystick or a module/circuitry for one or more of the same.
 19. Amethod, the method comprising: enabling the provision of differentialhaptic feedback based on the spatial interaction between a plurality ofuser interface elements associated with an electronic device.
 20. Acomputer program comprising computer program code, the computer programcode being configured to perform at least the following: enable theprovision of differential haptic feedback based on the spatialinteraction between a plurality of user interface elements associatedwith an electronic device.